Recursive architectures such as Tiny Recursive Models (TRMs) perform implicit reasoning through iterative latent computation, yet the geometric structure of these reasoning trajectories remains poorly understood. We investigate the activation manifold of TRMs during recursive unrolling and find that activations occupy an effectively linear, low-dimensional subspace whose principal directions can be tracked dynamically with cheap power iterations. This suggests that weight-sharing concentrates iterative computation along a small number of dominant eigendirections, and we find that this concentration varies sharply across computational sites. We exploit this structure through LASER (Low-Rank Activation SVD for Efficient Recursion), a dynamic compression framework that maintains an evolving low-rank basis via matrix-free subspace tracking with a fidelity-triggered reset mechanism, achieving ${\sim}60\%$ activation memory savings with no statistically significant accuracy degradation. Our analysis raises questions about how recursive architectures allocate representational capacity during implicit reasoning, and whether this concentration can be exploited to improve the efficiency and stability of latent computation.

Many applications seek to recover low-rank approximations of noisy tensor data.

Neural Information Processing Systems http://nips.cc/

The third entry varies under perturbation. Wecan compute the local indicator matrices atthis layer accordingly. We inherit the notations from the main text, and useIL to denote 13 theindicator matrixforlinearReLUoutputs. The key observation from this approach is that we can "merge" the weight matrices together for linearneurons(thefirstterminEq(19)).ThenwehavekW3D2LW2D1LW1k kW3kkW2kkW1k. Consider a neural network that maps inputx to output z = F(x), where z RN.

See Figure 1 for an illustration of the different SNR thresholds and Section 2.1 for more